In view of the small target of cigarette butts in surveillance videos of public places and the easy divergence of smoke generated by smoking， it is difficult to determine the smoking behavior only by target detection algorithm. Considering that the algorithm of posture estimation using skeleton key points is becoming more and more mature， a smoking behavior detection algorithm was proposed by using the relationship between human skeleton key points and smoking behavior. Firstly， AlphaPose and RetinaFace were used to detect the key points of human skeleton and face respectively. According to the ratio of distance between wrist and middle point of two corners of mouth and between wrist and the eye on the same side， a method for calculating whether the Smoking Action Ratio （SAR） in humans falls within the Golden Ratio of Smoking Actions （GRSA） to distinguish smoking from non-smoking behaviors was proposed. Then， YOLOv4 was used to detect whether cigarette butts existed in the video. The results of GRSA determination and YOLOv4 were combined to determine the possibility of smoking behavior in the video and make a determination of whether smoking behavior was present. The self-recorded dataset test shows that the proposed algorithm can accurately detect smoking behavior with the accuracy reached 92%.